Massively Parallel Algorithms for the Lattice Boltzmann Method on Non-uniform Grids

摘要

The lattice Boltzmann method exhibits excellent scalability on currentsupercomputing systems and has thus increasingly become an alternative methodfor large-scale non-stationary flow simulations, reaching up to a trillion gridnodes. Additionally, grid refinement can lead to substantial savings in memoryand compute time. These saving, however, come at the cost of much more complexdata structures and algorithms. In particular, the interface between subdomainswith different grid sizes must receive special treatment. In this article, wepresent parallel algorithms, distributed data structures, and communicationroutines that are implemented in the software framework waLBerla in order tosupport large-scale, massively parallel lattice Boltzmann-based simulations onnon-uniform grids. Additionally, we evaluate the performance of our approach ontwo current petascale supercomputers. On an IBM Blue Gene/Q system, the largestweak scaling benchmarks with refined grids are executed with almost two millionthreads, demonstrating not only near-perfect scalability but also an absoluteperformance of close to a trillion lattice Boltzmann cell updates per second.On an Intel-based system, the strong scaling of a simulation with refined gridsand a total of more than 8.5 million cells is demonstrated to reach aperformance of less than one millisecond per time step. This enablessimulations with complex, non-uniform grids and four million time steps perhour compute time.